1. Field of the Invention
The present invention relates to an image processing apparatus and image processing method. More particularly, the present invention relates to an image processing apparatus and image processing method capable of a quick response by omitting an unnecessary process while maintaining the order of image processes even when the image processes include one requiring a quick response.
2. Description of the Related Art
In recent years, image data are frequently manipulated with photo retouching applications which perform image modification, correction, and the like. For example, a photo retouching application may correct, adjust, and print captured image data, or upload captured image data to a web site in order to share it and/or make it open to the public.
Functions provided by the photo retouching application are basic correction and adjustment functions to improve poor quality of captured image data, for example, adjust the angle of view, brightness, and tint. In addition to these functions, the photo retouching application provides even a function of analyzing image data and automatically correcting it. FIG. 12 is a view showing the state transition of a general photo retouching application.
FIG. 12 shows a data sequence when the user selects an image to be corrected and inputs an instruction to correct the image data. In FIG. 12, particularly an exceptional process such as an error process is omitted. In correction instruction wait state S1201, an image is selected. In this state, the user has selected one image using an image selection UI (not shown), and is to select a correction process. When the user selects a correction process, the photo retouching application selects and executes an image process in accordance with his selection. In FIG. 12, four image processes are prepared for descriptive convenience. Image processes 1 to 4 are different from each other, and are, for example, automatic correction, automatic red-eye correction, sharpness correction, and brightness/contrast adjustment. Automatic correction provides a correction result suited to an image by analyzing an image without performing fine correction control by the user, such as correction of the brightness, color balance, and local brightness of image data.
For example, when image process 2 is red-eye correction, the photo retouching application changes from correction instruction wait state S1201 to image process 2 state S1203 to detect a red-eye area in image data and automatically perform red-eye correction. Alternatively, the photo retouching application prompts the user to designate a red-eye area, and corrects the designated red-eye area, providing the red-eye correction result to the user. After the end of the red-eye correction, the photo retouching application changes from image process 2 state S1203 to output state S1206 to output image data obtained in the image process 2 state. The image data is output by, for example, displaying the corrected image data on a display, printing it by a printer, or writing it on a recoding medium such as a CF card or HDD. After the end of the output, the photo retouching application changes from output state S1206 to correction instruction wait state S1201. At this time, the user can confirm the correction result on the display and input a new correction instruction. If the user is satisfied with the correction result, he can shift to selection of the next image, or input an instruction to, for example, end the photo retouching application.
FIG. 12 shows the state transition of a general photo retouching application. An application of this type has many independent functions, so image processes are often separately executed. Thus, this application assumes a user who is rather skilled in image correction.
However, the development of the image analysis technique and the use of the accompanying image correction technique can easily provide an effect which has not been obtained unless a plurality of correction effects are combined. As an example of the image analysis technique, a detection technique for an object, particularly a person's face in image data has been improved, and even a general PC can obtain a high-precision detection result. Further, even a photographing scene can be estimated from image data.
Many manufacturers have proposed automatic correction techniques capable of automatically obtaining optimum image correction effects for respective images using various image analysis results. Some of these automatic correction techniques can attain a very complicated correction result by combining correction for the entire image and locally controlled correction. An image correction application which provides a combination of automatic correction processes needs to control the order of image processes, instead of executing many image processes in random order by the user.
For example, the image correction application imposes restrictions such that automatic correction is always executed first, other correction processes are performed for an automatically corrected image, and double execution of automatic correction is inhibited. FIG. 13 is a view showing the state transition of image correction in the image correction application. In FIG. 13, particularly an exceptional process such as an error process is omitted. In FIG. 13, image processes 1 to 4 are executed sequentially in order named, unlike the photo retouching application shown in FIG. 12. In correction instruction wait state S1301, an image is selected. In this state, the user has selected one image using an image selection UI (not shown), and is to select a correction process. When the user selects a correction process, image processes 1 to 4 are sequentially executed in accordance with his selection. These image processes use, as input data, image data of the image selected by the user. The photo retouching application performs processes using, as an input, image data having undergone each image process. To the contrary, the image correction application uses uncorrected original image data.
Hence, when the user selects image processes 1 and 3 first, the image correction application executes image process 1 in image process 1 state S1302 using original image data as input data. Since image process 2 is not selected, the image correction application skips image process 2 state S1303, and executes image process 3 for output data in the image process 1 state in image process 3 state S1304. The image correction application skips subsequent image process 4 state S1305, outputs the result of executing image processes 1 and 3 in output state S1306, and then returns to correction instruction wait state S1301.
If the user is not satisfied with the obtained correction result, he inputs a new correction instruction in correction instruction wait state S1301. Assume that the user designates all image processes 1 to 4. In this case, the image correction application cancels the previous correction result, sequentially executes image processes 1 to 4 for original image data, and outputs the result.
Some image processes designate a plurality of correction areas. An example of this process is a manual red-eye correction function of designating a correction area by the user and performing red-eye correction. Such a function holds user area designation information, and controls the image process to reflect the correction results of all designated areas every time area designation information is added.
Japanese Patent Laid-Open No. 2009-164972 provides a system in which an image process is described in advance as a predetermined template pattern. The user can create a modified template based on the predetermined template. The system allows the user to freely control the process (for example, contents and parameters).
In the foregoing image processing application, the order of image processes is determined. When providing an image process sequence in which the order of image processes is determined, the process needs to be executed from the beginning in the process sequence as shown in FIG. 13 every time the image process instruction changes. Since even designated correction is executed every time one correction instruction changes, no high-speed process can be implemented. Especially, manual adjustment (1D-LUT process for color balance, brightness, and CMYD) and the like are often provided to the user via a UI (User Interface) as shown in FIG. 14. In FIG. 14, a manual adjustment function is selected. Two functions, that is, manual adjustment 1 U1402 and manual adjustment 2 U1404 are provided as buttons, and the function is enabled by pressing the button. For descriptive convenience, manual adjustment 1 is color balance correction, and manual adjustment 2 is brightness correction. As a matter of course, UIs and functions are not limited to them. In color balance correction, the tint of image data is corrected. A slider bar U1403 is moved rightward to enhance the blueness of the entire image, and leftward to enhance the redness. In brightness correction, the brightness of image data is adjusted. A slider bar U1405 is moved rightward to correct the entire image to be bright, and leftward to correct it to be dark.
With the manual adjustment function, the user desirably confirms the correction effect and determines the correction amount while visually checking a preview image (not shown). For the manual adjustment function, a quick response is important to reflect the process result at the same time as the UI operation. When correction functions are provided by classifying them into a plurality of categories as UIs, like correction 1 U1406 and correction 2 U1407, it is very important for user friendliness to reflect the correction effect as soon as possible. This also applies to a case in which the correction operation is performed by switching the UI.
Although the response speed of the correction process is important, it is difficult to increase the response speed in control of executing image processes in predetermined order, as shown in FIG. 13. Japanese Patent Laid-Open No. 2009-164972 provides a system which allows the user to describe a process sequence. However, this reference does not describe a dynamic change of the process sequence to optimize internal processes and increase the process speed.